Method and system for predicting a trajectory of a target vehicle in an environment of a vehicle

ABSTRACT

A method for predicting a trajectory of a target vehicle in an environment of a vehicle. The method includes the steps of a) capturing states of the target vehicle, capturing states of further vehicle objects in the environment of the vehicle and capturing road markings by a camera-based capture device; b) preprocessing the data obtained in step a), wherein outliers are removed and missing states are calculated; c) calculating an estimated trajectory by a physical model on the basis of the data preprocessed in step b); d) calculating a driver-behavior-based trajectory on the basis of the data preprocessed in step b); and e) combining the trajectories calculated in steps c) and d) to form a predicted trajectory of the target vehicle.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to German Patent Application No. 102020 117 004.1, Jun. 29, 2020, the content of such application beingincorporated by reference herein in its entirety.

FIELD OF THE INVENTION

The present invention relates to a method and a system for predicting atrajectory of a target vehicle in an environment of a vehicle.

BACKGROUND OF THE INVENTION

Modern motor vehicles often have driver assistance systems (“AdvancedDriver Assistance Systems”, ADAS), for example adaptive cruise controldevices, collision warning systems, emergency braking assistants,highway assistants or congestion assistants, in order to reduce the riskof vehicle collisions and to increase driving comfort. Such driverassistance systems which zo operate in a partially autonomous orautonomous manner require a prediction of a trajectory of a targetvehicle in the environment of the vehicle. Such a trajectory predictionis used, for example, as a basis for functional decisions, routeplanning or braking interventions. A predicted trajectory typicallyconsists of a number N of predicted states of the target vehicle in twodimensions, for example the position, the speed, the yaw and the yawrate of the target vehicle.

The methods known from the prior art for predicting a trajectory of atarget vehicle in an environment of a vehicle use very differentphysical models or driving-maneuver-based models which must be adaptedin a relatively complicated manner.

EP 3 467 799 A1, which is incorporated by reference herein, discloses amethod for predicting a trajectory of a target vehicle in an environmentof a vehicle, in which the type of vehicle moving along a lane of a roadis captured and an item of movement prediction information is generatedin order to predict a movement of the target vehicle on the basis of thetype of target vehicle. In this case, the movement is assigned to thelane.

A challenge when predicting a trajectory of a target vehicle involves,inter alia, predicting the states of the target vehicle, in particularthe position, the speed, the yaw and the yaw rate, for a period of up tofive seconds in advance on the basis of the sensor signals ormeasurement data provided by a camera-based capture device. The methodfor predicting a trajectory of a target vehicle in an environment of avehicle should also be as robust as possible in order to minimize theinfluence of outliers which are caused, for example, by random orsystematic errors in the measured states, in particular in the positionsand speeds of the target vehicle. On the basis of this, described hereinis a further improved method and a system for predicting a trajectory ofa target vehicle in an environment of a vehicle.

SUMMARY OF THE INVENTION

A method according to aspects of the invention for predicting atrajectory of a target vehicle in an environment of a vehicle comprisesthe steps of:

a) capturing states of the target vehicle, capturing states of furthervehicle objects in the environment of the vehicle and capturing roadmarkings by means of a camera-based capture device,

b) preprocessing the data obtained in step a), wherein outliers areremoved and missing states are calculated,

c) calculating an estimated trajectory by means of a physical model onthe basis of the data preprocessed in step b),

d) calculating a driver-behavior-based trajectory on the basis of thedata preprocessed in step b),

e) combining the trajectories calculated in steps c) and d) to form apredicted trajectory of the target vehicle.

The method according to aspects of the invention makes it possible topredict the states of the target vehicle, in particular the position,the speed, the yaw and the yaw rate, preferably for a period of up tofive seconds in advance on the basis of the data provided by thecamera-based capture device. A robust method for predicting a trajectoryof a target vehicle in an environment of a vehicle is also provided inorder to minimize the influence of outliers which can be caused, forexample, by random or systematic errors in the measured states, inparticular in the positions and speeds of the target vehicle. In thiscase, the prediction of the trajectory of the target vehicle mayadvantageously take into account physical and environmental aspects ofthe respective driving situation and may weight them accordingly.

In one preferred embodiment, it is proposed that the trajectoriescalculated in steps c) and d) are optimized by means of an optimizationalgorithm before being combined. A simulated annealing algorithm maypreferably be used as the optimization algorithm.

In one embodiment, it is possible to use a RANSAC filter algorithm topreprocess the data obtained in step a). This makes it possible toreliably capture and eliminate possible outliers in the measurementdata.

A practical problem is often the fact that the camera sensors of thecamera-based capture device is normally cannot provide a reliableestimation of the yaw and the yaw rate of the target vehicle. This isbecause the target positions of the target vehicle which are captured bymeans of the camera-based capture device are typically accompanied byboth white, Gaussian noise and non-white, non-Gaussian noise. In oneadvantageous embodiment, it is therefore proposed that an alpha-betafilter algorithm is used to remove noise components from the dataobtained in step a).

In one advantageous development, provision may be made for the course ofthe road and the course of the lane to be estimated when preprocessingthe data obtained in step a) on the basis of measured states of thetarget vehicle and of further vehicle objects in the environment and ofthe vehicle itself and on the basis of the lane markings and staticobjects captured by the camera-based capture device.

In one embodiment, it is possible to use a modified CYRA model tocalculate the estimated trajectory by means of the physical model instep c), in which model individual trajectories are calculated in aparallel manner by means of a plurality of physical models and aweighted trajectory is calculated by combining the individualtrajectories. In this case, the modified CYRA model is based on theassumption of a constant yaw rate and a constant acceleration(CYRA=“Constant Yaw Rate and Acceleration”).

A system for carrying out the above described method comprises acamera-based capture device, which is designed to capture states of thetarget vehicle, to capture states of further vehicle objects in theenvironment of the vehicle and to capture road markings according tostep a), and a computing device, in which at least means for carryingout steps b) to e) of the method are implemented.

In one embodiment, provision may be made for the computing device tocomprise a prediction module in which a driver behavior classificationmodule, in particular, is implemented, which has a Markov state machinein addition to a driving maneuver detection system. This makes itpossible to improve the behavior classification. In this case, thebehavior of the target vehicle is classified in a plurality ofcategories, for example “lane change”, “keep in lane”, “accelerate”,“brake”, “maintain speed”, on the basis of a normalized lane assignmentof the target vehicle, the derivative of the normalized lane assignmentof the target vehicle and the speed vector. The input variables of thedriver behavior classification module preferably form a number N ofestimated historical states of the target vehicle, a number N ofestimated historical states of the other vehicle objects and theestimated course of the road.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

Further features and advantages of the present invention become clear onthe basis of the following description of a preferred exemplaryembodiment with reference to the accompanying figures, in which

FIG. 1 shows a schematic illustration of a system which is designed tocarry out a method for predicting a trajectory of a target vehicle in anenvironment of a vehicle according to one preferred exemplary embodimentof the present invention,

FIG. 2 shows a schematic illustration which illustrates details of thepreprocessing of the data captured by a camera-based capture device ofthe system,

FIG. 3 shows a schematic illustration which shows details of anoptimization algorithm.

DETAILED DESCRIPTION OF THE INVENTION

With reference to FIG. 1, a system 1 for carrying out a method forpredicting a trajectory of a target vehicle in an environment of avehicle comprises a camera-based capture device 2 which is designed, inparticular, to capture and track the target vehicle and further vehicleobjects in the environment of the vehicle and to detect lane markings.

The system 1 also comprises a computing device 3, in which a trajectoryprediction module 4 is implemented, which module receives themeasurement data from the camera-based capture device 2 as inputvariables and is designed to predict the trajectory of the targetvehicle in the environment of the vehicle on the basis of thesemeasurement data in the manner explained below by means of acomputer-implemented method. For this purpose, a plurality of softwaremodules 40, 41, 42, 43, 44, the functions of which shall be explained inmore detail below, are implemented inside the trajectory predictionmodule 4.

The prediction module 4 has a preprocessing module 40 which is designedto preprocess the data which are captured by the camera-based capturedevice 2 and are made available to the prediction module 4 for furtherprocessing. The preprocessing module 40 is configured to remove is anyoutliers in the measured states of the target vehicle and of the furthervehicle objects from the measurement data by means of an accordinglydesigned data preprocessing algorithm and to calculate missing states ofthe target vehicle and of the further vehicle objects.

In many situations, no lane markings, which can then be processedfurther, may be captured with the required reliability by thecamera-based capture device 2 during the journey of the vehicle.Therefore, an algorithm is also implemented in the preprocessing module40 and is designed to estimate the course of the road and the course ofthe lane on the basis of measured states of the target vehicle and offurther vehicle objects in the environment and of the vehicle itself andon the basis of the lane markings and static objects, for exampleguardrails, gradients, etc., captured by the camera-based capture device2.

A physical trajectory calculation module 41 is also implemented in theprediction module 4 and can calculate the trajectory of the targetvehicle in the environment of the vehicle from the data provided by thepreprocessing module 40 on the basis of a physical trajectorycalculation model. In this case, a modified CYRA model is preferablyused, in which model individual trajectories of the target vehicle arecalculated in a parallel manner by means of a plurality of physicalmodels and a weighted combination of these individual trajectories iscalculated. In this case, the modified CYRA model is based on theassumption of a constant yaw rate and a constant acceleration (“ConstantYaw Rate and Acceleration”, CYRA for short) of the target vehicle.

A driver behavior classification module 42 is also implemented in theprediction module 4 and, in addition to a driving maneuver detectionsystem, has a Markov state machine in order to improve the behaviorclassification. In this case, the behavior of the target vehicle isclassified in a plurality of categories, for example “lane change”,“keep in lane”, “accelerate”, “brake”, “maintain speed”, on the basis ofa normalized lane assignment of the target vehicle, the derivative ofthe normalized lane assignment of the target vehicle and the speedvector. Input variables of the driver behavior classification module 42form a number N of estimated historical states of the target vehicle, anumber N of estimated historical states of the other vehicle objects andthe estimated course of the road.

A path planning module 43 is also implemented in the prediction module4, which path planning module receives input data from the driverbehavior classification module 42 and is designed to predict and outputa behavior-based trajectory of the target vehicle. The path planningmodule 43 is configured to calculate a driver-behavior-based trajectoryof the target vehicle on the basis of a is behavior categoryrepresenting the driver behavior and on the basis of the target vehiclestates and the target vehicle history.

A trajectory combining module 44 is also implemented in the predictionmodule 4. This trajectory combining module 44 is designed to combine theparameters of the trajectory, which is calculated by means of thephysical trajectory calculation module 41, and the parameters of thetrajectory, which is calculated by means of the path planning module 43,on the basis of an optimization algorithm. For this purpose, anoptimization module 5 is implemented in the computing device 3 and isdesigned to jointly adapt all parameters of the trajectory of the targetvehicle, which are obtained by means of the physical trajectorycalculation module 41, the driver behavior classification module 42 andthe path planning module 43, by means of appropriate optimizationtechniques.

With reference to FIG. 2, further details of the preprocessing of thedata captured by the camera-based capture device 2 by means of thepreprocessing module 40 shall be explained in more detail below. In thiscase, use is made of a novel method for estimating the yaw and the yawrate of the target vehicle which is based on a number N (buffer 400) ofprevious target positions (x, y positions) of the target vehicle.

A substantial challenge is the fact that the camera sensors of thecamera-based capture device 2 normally cannot provide a reliableestimation of the yaw and the yaw rate of the target vehicle. This isbecause the target positions of the target vehicle which are captured bymeans of the camera-based capture device 2 are accompanied by bothwhite, Gaussian noise and non-white, non-Gaussian noise.

In the method used here, the target yaw and the target yaw rate areestimated by a yaw calculation module 402 on the basis of camerameasurements, wherein outliers which are preferably captured by anadaptive RANSAC algorithm 401 are disregarded and noise influences areeliminated, preferably by means of an alpha-beta filter algorithm 403.In addition to the number N of previous target positions (x, ypositions) of the target vehicle from the buffer 400, the speed of thetarget vehicle in the x direction is also included in the RANSACalgorithm 401.

The camera-based capture device 2 outputs the vehicle states of thetarget vehicle as positions in the x-y coordinate system of the vehicle,wherein the x axis represents the forward direction of travel of thevehicle. The novel algorithm for estimating the yaw and the yaw rate isbased on a is modified RANSAC filter algorithm 401 and an alpha-betafilter 403. In this case, a number N of measurements of the targetpositions of the target vehicle is recorded. The positions of the targetvehicle are fitted, with the aid of the RANSAC algorithm 401, with aline, the parameters of which are adapted online on the basis of thetarget position and the target speed. The angle between the fitted lineand the x coordinate of the target vehicle can be calculated very easilyand forms a measure of the yaw of the target vehicle. The yaw calculatedin this manner is made available to an alpha-beta algorithm 403 which isdesigned to output filtered yaw and an estimated yaw rate.

With further reference to FIG. 3, further details of the optimizationmethod which is carried out by means of the optimization module 5 shallbe explained in more detail below.

Object data relating to the target vehicle and relating to furthervehicle objects in the environment of the vehicle and detected roadmarkings are recorded by means of the camera-based capture device 2. Inthis case, each target vehicle and each further vehicle object areprovided as two-dimensional boxes with their corresponding covariancematrix by means of a camera interface of the camera-based capture device2. Each captured road marking is provided as a clothoid parameter viathe camera interface.

The optimization module 5 has a reference extraction module 50 whichprocesses the data obtained in the manner described above from thecamera-based capture device 2 as follows:

-   -   Recorded road markings (road lines), which were recorded as        clothoids, are fitted with a cubic polynomial.    -   The road markings (road lines) are supplied to a novel algorithm        for extracting the course of the road, by means of which an        estimation of the course of the road is calculated.    -   Recorded positions of the target vehicle and of the further        vehicle objects are processed by means of a smoothing function.        The result of this smoothing is an estimation of the real        trajectories of the target vehicle and of the further vehicle        objects.    -   The basic truth behavior of the driver is characterized on the        basis of the estimation of the course of the road and the        trajectories of the target vehicle and the further vehicle        objects.

These data are made available to an evaluation module 51. The predictionmodule 4 reads in the is data recorded by the camera-based capturedevice 2 and, after preprocessing these data, provides a prediction fora trajectory of a particular target vehicle. Errors in the x and ycoordinates between the prediction and the estimated basic truthbehavior of the driver are calculated by means of the evaluation module51. The difference to the driver behavior classified by means of thedriver behavior classification module 42 is compared with thecharacterized basic truth behavior of the driver.

An optimization algorithm 52 which is preferably based on a simulatedannealing method calculates updated parameters for the prediction module4 in order to minimize errors in the x and y directions and errors inthe driver behavior classification module 42.

The algorithm for extracting the course of the road provides a novel wayof extracting the estimation of the actual course of the road fromcamera recordings of the camera-based capture device 2 using imprecisemeasurements of road markings. This algorithm uses a window withrecorded, imprecise and relatively short road markings (lines) anddetermines the actual position of the road marking therefrom usinghistogram estimations. In addition to the histogram estimation, thealgorithm preferably also uses heuristics in order to detect merging andjunction points of the road. The output of this algorithm is anestimated course of the road.

What is claimed is:
 1. A method for predicting a trajectory of a targetvehicle in an environment of a vehicle, said method comprising the stepsof: a) capturing states of the target vehicle, capturing states offurther vehicle objects in the environment of the vehicle, and capturingroad markings using a camera-based capture device, b) preprocessing thedata obtained in step a) by removing outliers and calculating missingstates, c) calculating an estimated trajectory using a physical model onthe basis of the data preprocessed in step b), d) calculating adriver-behavior-based trajectory on the basis of the data preprocessedin step b), and e) combining the trajectories calculated in steps c) andd) to form a predicted trajectory of the target vehicle.
 2. The methodas claimed in claim 1, further comprising optimizing the trajectoriescalculated in steps c) and d) using an optimization algorithm before thetrajectories calculated in steps c) and d) are combined.
 3. The methodas claimed in claim 2, further comprising using a simulated annealingalgorithm as the optimization algorithm.
 4. The method as claimed inclaim 1, further comprising using a RANSAC filter algorithm topreprocess the data obtained in step a).
 5. The method as claimed inclaim 1, further comprising using an alpha-beta filter algorithm toremove noise components from the data obtained in step a).
 6. The methodas claimed in claim 1, further comprising estimating a course of theroad and a course of the lane when preprocessing the data obtained instep a) on the basis of measured states of the target vehicle and offurther vehicle objects in the environment and of the vehicle itself andon the basis of the lane markings and static objects captured by thecamera-based capture device.
 7. The method as claimed in claim 1,further comprising using a modified CYRA model to calculate theestimated trajectory using the physical model in step c), in whichmodified CYRA model (i) individual trajectories are calculated in aparallel manner using a plurality of physical models and (ii) a weightedcombination of the individual trajectories is calculated.
 8. A systemfor predicting a trajectory of a target vehicle in an environment of avehicle, said system comprising a camera-based capture device and acomputing device, wherein the system is configured to predict atrajectory of a target vehicle in an environment of a vehicle by: a)capturing states of the target vehicle, capturing states of furthervehicle objects in the environment of the vehicle, and capturing roadmarkings using the camera-based capture device, b) preprocessing thedata obtained in step a) using the computing device by removing outliersand calculating missing states, c) calculating an estimated trajectoryusing a physical model on the basis of the data preprocessed in step b),d) calculating a driver-behavior-based trajectory on the basis of thedata preprocessed in step b), and e) combining the trajectoriescalculated in steps c) and d) using the computing device to form apredicted trajectory of the target vehicle.
 9. The system as claimed inclaim 8, wherein the computing device comprises a prediction module inwhich a driver behavior classification module is implemented, which hasa Markov state machine in addition to a driving maneuver detectionsystem.